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Abstract 

Rapid,  on-line  trajectory  generation  for  reusable  launch  vehicles  is  gaining  high  interest  with  the 
explosive  desire  for  more  intelligent  and  autonomous  systems.  There  are  many  aspects  that 
make  such  achievement  very  appealing,  such  as  being  able  to  autonomously  re-plan  a  mission 
on-the-fly  in  the  case  of  abort  guidance.  Due  to  the  constrained,  highly  nonlinear  nature  of  the 
reentry  problem,  feasible  and  computationally- efficient  guidance  and  control  methods  have  been 
a  challenge  since  the  beginning  of  the  space  age.  This  paper  demonstrates  the  feasibility  of  using 
a  new,  on-line  sampled-data  feedback  control  algorithm  for  fast  and  accurate  nonlinear  optimal 
closed- loop  control  of  a  generic  reentry  vehicle  in  the  presence  of  disturbances.  A  direct 
Legendre  psueodospectral  method  is  used  for  the  rapid  re- computation  of  the  open- loop  optimal 
controls  such  that  a  successive  optimal  solution  is  fed  back  to  the  system  to  help  manage 
uncertainties  and  external  disturbances.  The  method  is  based  on  a  receding- horizon  control 
approach,  but  cbes  not  require  any  prior  knowledge  of  the  computation  time.  Successful 
performance  of  this  approach,  combined  with  a  spectral  algorithm,  shows  that  rapid  trajectory 
generation  is  not  only  achievable,  but  that  it  is  fast  enough  to  provide  optimal  closed- loop 
feedback  with  successive  re- computations. 
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PREPRINT 


1.  Introduction 

With  ongoing  efforts  to  improve  the  safety,  reliability,  and  cost  of  reusable  launch  vehicle  (RLV) 
systems  and  operations  along  with  the  more  recent  focus  on  a  U.S  Space  Shuttle  replacement 
following  its  retirement  around  2012,  there  is  a  new  opportunity  to  implement  advanced,  yet 
simpler  and  more  effective  guidance  and  control  methods  than  used  in  previous  launch 
technologies.  A  key  objective  that  is  sought  in  this  endeavor  as  well  as  almost  all  other  sectors 
of  technology  is  the  requirement  for  more  intelligent  systems  that  can  operate  autonomously 
without  as  much  human  interaction.  Of  course  this  has  the  underlying  stipulation  that  they 
operate  safer,  more  reliable,  and  more  efficiently  than  before.  To  accomplish  this  objective  for 
the  case  of  a  RLV,  the  guidance  and  control  during  ascent,  reentry,  or  landing  needs  to 
autonomously  happen  on-line  and  in  real-time.  The  RLV  intelligent  control  (i.e.  “brains”)  needs 
to  be  capable  of  adapting  to  rapidly  changing  circumstances,  handle  large  external  disturbances, 
large  parameter  uncertainties,  re-generate  trajectories  (i.e.  re-plan  how  to  get  to  a  designated  or 
alternate  landing  site),  reconfigure  its  controls  in  the  advent  of  an  unforeseen  control  failure  and 
then  of  course  to  figure  out  how  to  allocate  the  controls. 

Originally,  the  challenge  of  solving  constrained,  highly  nonlinear  dynamics  and  the  limitations  of 
computational  speed  at  the  time,  demanded  linearized,  approximate  equations.  Often,  the  control 
equations  were  driven  with  tediously  derived  gain- scheduled  schemes  such  as  the  drag-based 
tracking  techniques  used  in  Space  Shuttle  entry  guidance  where  a  series  of  drag  reference 
segments  required  individual  controller  gain  design  [1], 

With  the  realization  of  advances  in  computational  power  and  numerical  algorithms,  recent 
research  efforts  for  solving  the  reentry  problem  have  focused  on  real-time,  on-line  trajectory 
generation,  guidance  adaptation,  and  control  reconfiguration.  Oppenheimer,  Bolender,  and 
Doman  at  the  Air  Force  Research  Lab’s  office  of  Advanced  Guidance  and  Control  have  done 
extensive  work  on  reconfigurable  control  in  the  form  of  an  optimal  control  allocation  algorithm 
for  such  vehicles  as  the  X-40,  X-37,  and  X-33  [2]-[4],  Reconfiguration  capabilities  for  RLV 
guidance  and  control  systems  has  also  been  rigorously  pursued  by  others  mainly  in  support  of 
two  program  initiatives  to  support  such  work:  Marshall  Space  Flight  Center’s  (MSFC)  Advanced 
Guidance  and  Control  (AG&C)  Program  and  the  Air  Force  Research  Lab’s  (AFRL)  Integrated 
Adaptive  Guidance  and  Control  (IAG&C)  Program.  More  on  these  programs  can  be  found  in 
Ref.  [5]  and  [6], 

Since  control  reconfiguration  and  guidance  adaptation  will  not  always  be  enough  to  recover  a 
vehicle  after  control  failures,  Schierman  et  al,  has  taken  this  one  step  further  by  developing  an 
architecture  that  allows  for  reshaping  of  the  outer- loop  guidance  trajectory  commands  and 
retargeting  an  alternate  landing  site  should  the  nominal  or  original  mission  be  aborted  [6,7]. 
Likewise,  Shaffer  has  integrated  this  need  of  trajectory  reshaping  and  retargeting  with  the 
reconfigurable  control  work  of  Oppenheimer  et  al.,  to  demonstrate  relatively  fast  computations 
of  optimal  trajectories  under  trim  deficient  path- constraints  [8], 

Moving  away  from  the  successful,  but  extremely  time-consuming  gain- scheduling  approach  for 
flight- control  designs,  numerous  efforts  have  employed  a  form  of  feedback  linearization,  often 
referred  to  as  dynamic  inversion.  Through  the  use  of  nonlinear  feedback,  dynamic  inversion 
essentially  cancels  the  nonlinearities  of  the  system  to  form  an  augmented  linear  system  that  can 
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then  be  controlled  using  more  traditional  linear  control  techniques.  Unfortunately,  this 
linearization  technique  requires  exact  knowledge  of  the  system  model,  such  as  specific  plant 
parameters.  For  air  vehicles  where  large  uncertainties  such  as  aerodynamics  forces  and  moments 
are  inevitable,  this  method  could  easily  result  in  unstable,  unrecoverable  flight  conditions.  For 
this  reason,  other  nonlinear  feedback  methods  are  being  investigated.  Some  methods  include 
various  combinations  of  adaptive  control,  backstepping,  and  even  more  robust  versions  of 
dynamic  inversion. 

One  way  to  address  the  problem  is  by  using  an  optimal  trajectory  generator  to  solve  for  a 
reference  input  trajectory  off-line,  and  then  use  other  inner- loop  control  means  to  track  the 
desired  trajectory.  Schierman  et  al.  has  recently  demonstrated  the  use  of  an  outer- loop  optimal 
control  scheme  to  generate  a  reference  trajectory  that  is  tracked  by  an  LQR  inner- loop  controller 
[9],  In  a  similar  fashion,  Carson  uses  an  outer- loop  guidance  scheme  with  LQR  inner- loop 
feedback  to  manage  disturbances  and  uncertainty  [10].  Carson’s  work  employs  a  method 
presented  by  Milam  [11],  whereby  real-time  trajectory  generation  is  made  possible  by  finding  the 
trajectory  curves  in  a  lower  dimensional  space,  parameterizing  the  curves  with  B-splines,  and 
then  using  Sequential  Quadratic  Programming  (SQP)  to  find  the  spline  coefficients  that  satisfy 
the  optimization  objectives  and  constraints. 

With  online  approaches  becoming  more  capable  of  solving  highly  nonlinear  systems,  there  is 
really  no  need  for  off-line  designs  of  closed- loop  feedback  laws.  In  this  paper,  an  intelligent 
trajectory  guidance  system  is  designed  for  a  generic  reentry  vehicle  by  solving  the  optimal 
control  problem  online  without  an  inner-  loop  tracking  controller.  Once  provided  the  initial  and 
final  conditions  for  the  reentry  vehicle,  the  proposed  algorithm  computes  the  nonlinear  optimal 
control  As  in  most  control  designs,  feedback  is  required  to  manage  uncertainties.  For  the 
reentry  problem,  parameter  uncertainty  and  external  disturbances  such  as  wind  gusts  (or  shear) 
cause  the  vehicle  to  deviate  from  its  nominal  trajectory  that  may  have  been  the  originally 
predicted/computed/planned  optimal  trajectory.  The  proposed  guidance  and  control  method, 
adapted  from  the  promising  work  of  Pooya,  Fleming,  and  Ross  on  a  spacecraft  optimal- time  slew 
maneuver  [12],  uses  a  sampled-data  feedback  law  to  provide  an  optimal  trajectory  in  the 
presence  of  uncertainty  and  disturbances.  Rather  than  track  a  pre-computed  solution,  the  scheme 
re- solves  the  optimal  control  problem  and  updates  the  control  command  as  soon  as  a  new 
solution  is  calculated.  This  method  is  also  similar  to  Schierman’s  Optimal-Patb-to-Go  (OPTG) 
Trajectory  Reshaping  Strategy  presented  in  his  early  2001  work  [13],  The  work  presented  herein 
differs  from  Schierman’s  in  that  it  is  primarily  used  for  feedback  without  using  an  hner-loop 
tracking  controller.  The  original  concept  dates  back  to  the  early  1990’s,  when  Pesch  discussed 
off-line  and  on-line  methods  for  aerospace  applications  [14],  He  introduced  the  combination  of 
what  he  called  the  “Neighboring  Optimal  Feedback  Guidance”  with  a  “Repeated  Correction 
Guidance  Scheme”  to  essentially  perform  the  same  objective  as  the  method  presented  in  Pooya’ s 
work  [12].  The  main  difference  is  that  his  method  relied  on  a  linearized  indirect  multiple 
shooting  technique  that  requires  a  more  complicated  implementation  at  increased  computational 
cost.  Unlike  Pesch’ s  “repeated  correction  method,”  the  method  presented  in  this  paper 
guarantees  that  the  constraints  are  satisfied  before  feeding  back  the  control  signal  provided  there 
is  a  feasible  solution  .  Also,  it  was  claimed  that  the  “theoretical  and  numerical  basis”  for  the  on¬ 
line  approach  is  not  mature  enough  for  general  optimal  control  problems.  Although  this  paper 


3 


does  not  provide  the  “mathematical  justification”  that  Pesch  desires,  it  is  yet  another  stepping- 
stone  to  making  on-line  trajectory  optimization  more  practical. 

To  solve  the  optimal  control  problem,  a  spectral  algorithm  [15-18]  known  as  the  Legendre 
Pseudospectral  Method  is  employed  by  use  of  a  MATLAB-based  software  package  called  DIDO 
[19].  This  direct  method  discretizes  the  problem  and  approximates  the  states,  co-states  and 
control  variables  by  use  of  Lagrange  interpolating  polynomials  where  the  unknown  coefficient 
values  coincide  with  the  Legendre-Gauss-Lobatto  (LGL)  node  points.  After  this  approximation 
step,  an  NLP  solver  (SNOPT)  solves  a  sequence  of  finite-dimensional  optimization  problems 
that  capture  the  nonlinearities  of  the  system  in  the  form  of  an  optimal  control.  For  an  extensive 
description  of  this  method  and  its  use  for  real-time  optimal  control,  see  references  [15]- [21], 

With  a  lot  of  interest  in  real-time  trajectory  generation,  this  paper  intends  to  demonstrate  that  by 
using  a  fast  psuedospectral  optimization  method,  not  only  are  rapid  optimal  trajectories 
achievable,  but  through  successive  trajectory  generations  as  a  nonlinear  sampled-data  feedback 
law,  optimal  guidance  and  control  is  attained  in  the  presence  of  parameter  uncertainty  and 
external  disturbances. 

2.  Problem  Definition  &  Formulation 

2.1  Reentry  Model  (Kinematics  and  Dynamics) 

Using  a  reduced-order  model  is  all  that  is  required  to  clearly  demonstrate  that  the  nonlinear 
feedback  technique  is  an  effective  method  towards  reentry  trajectory  optimization.  Therefore, 
for  this  paper,  the  full  6- DOF  equations  of  motion  are  simplified  and  decoupled  to  create  a 
reduced-order  model.  The  model  assumes  a  point- mass- model  over  a  flat,  non- rotating  earth 
such  that  the  positional  and  translational  equations  of  motion  in  a  Cartesian  “local  horizontal” 
coordinate  system  become 


x  =  V  cosy  cosE, 
y  =  V cosy  sin^ 
z  =  V  sin  y 


V=- 


y  = 


div,z} 

m 

L{V,z} 


g  smy 
gcosy 


$  = 


mV  V 
L{V,  zjsina 


mV  cosy 


(1) 


Where  x  (down- range),  y  (cross-range),  and  z  (altitude)  are  the  vehicle’s  position  with  respect 
to  the  fixed-earth  reference  frame,  V  is  the  velocity  magnitude  (i.e.  total  airspeed),  y  is  the 

flight-path- angle  (FPA),  £,  is  the  heading  angle,  a  is  the  angle -of- attack  (AoA),  and  a  is  the 
angle-of-bank  (AoB). 
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The  lift  and  drag  forces  are  represented  as  L  and  D,  respectively,  given  by 


L=±p{z}V2CL{a}Sref  - ^-( p0e~z/z° )V2CL {a }Sref  (2) 

D  =  ^p{z}V2CD{a}Sref  =^(p0e-^)V2CD{a}Sref  (3) 


2  1  2  _ 

where  Sref  is  the  reference  area  given  as  1600  ft  ,  the  term  —p(z)V  is  the  dynamic  pressure,  q  , 

and  the  term  p0<?  2  30  is  the  commonly  accepted  two-parameter  model  for  atmospheric  density, 
p  ,  as  a  function  of  altitude.  This  density  approximation,  where  po  is  the  reference  density  taken 
to  be  1.725  kg/m3  (0.003399  slugs/ft3),  z  is  the  current  altitude,  and  Z)  is  the  atmospheric  scale 
height  (i.e.  reference  height)  taken  to  be  6700  m  (21981.6  ft),  is  good  for  closed- form  solutions 
and  fits  well  from  5  to  40  km  in  altitude  [22], 

For  simplicity,  the  coefficients  of  lift  (Cl)  and  drag  (Cd)  are  modeled  as 

Cl\a\^C,+Cla  (4) 

=  ^ dq  ^ 

Where  the  coefficients  are 

Cu  =-0.2070,  Cu  =0.0292,  CA  =0.0785,  CDa  =  -0.6159xl0-2,  and  CD^  =  0.6214xl0"3  with 
a  in  degrees  [23]. 

To  fully  capture  the  state  at  any  instant,  the  state  vector  for  this  reduced- order  model  is  given  as 

x  =  [x  y  z  V  y  ^  f  e  M6  (6) 

Without  a  thrust  force  for  the  reentry  gliding  problem,  the  only  controllable  parameters  for  this 
reduced-order  model  are  the  lift  and  drag  forces.  Typically,  for  symmetric  flight  (i.e. 
coordinated  turns  where  |3  =  0 ),  the  lift  and  drag  coefficients  can  be  determined  by  the  vehicle’s 
angle- of- attack  and  Mach  number,  a  function  of  velocity  and  speed- of- sound  at  a  given  altitude. 
However,  it  is  the  physical  modulation  of  angle- of- attack  and  bank  angle  that  controls  the 
vehicle’s  translational  motion  through  x-y-z  space.  Therefore,  a  common  control  vector  for  the 
reentry  problem  is 


n=[ a  ofeR2 

Of  course  with  these  control  variables  defining  the  control  vector,  it  is  assumed  that  there  are  no 
command  delays  (i.e.  lags),  hence;  this  type  of  control  is  sometimes  referred  to  as  “inertialess” 
control.  To  help  compensate  for  this  and  add  a  little  realism  to  the  problem,  as  explained  in 
references  [8]  and  [23],  a  new  control  vector  is  formed  such  that  rate  limits  can  be  modeled  as 
well.  This  is  accomplished  by  using  “virtual”  controls  [23]  mathematically  expressed  as 
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u  =  [ua  u  f  e  R2 

_  (8) 

Now,  since  the  algorithm  uses  these,  the  original  state  vector  must  include  the  physical  controls, 
a  and  a  ,  to  form  a  new  state  vector: 

x  -  [x  y  z  V  y  \  a  a  f  €  R8 

Remark:  For  a  real  vehicle,  or  in  a  6-DOF  simulation,  it  is  the  control  surface  deflections  that 
create  body  moments  and  forces  to  augment  the  wind-relative  angle -of- attack,  bank  angle,  and 
sideslip  angle. 


2.2  General  Problem  Formulation 

As  with  any  dynamical  optimization  problem,  the  cost  function  (a.k.a  objective  function), 
governing  equations  of  motions,  path  constraints,  boundary  limits  on  initial/final  conditions,  and 
any  constraints  (on  states  and/or  controls)  must  be  defined.  As  such,  the  general  optimal  control 
problem  for  trajectory  generation  is  fully  posed  in  the  following  manner: 


min„  j(x(^),M.(x),x0,xf)=E(x(x0),x(tf),x0,xf)+jF(x(z),u(x),x)dx 
subject  to  x  =  f  (x,u,x) 
h  -h.(x,u,x ) 

eiZe(x{T0),x(xf),T  od/)^ 

X, 

M,  <m(t)  <«„ 

The  goal  is  to  find  a  state-control  function  pair,  {x (•),«(•)},  or  sometimes  time,  x,  that 
minimizes  the  performance  index  represented  by  the  Bolza  form,  /(•),  consisting  of  either  a 
Mayer  term,  £’(•),  a  Lagrange  term,  F(  ),  or  both  as  stated  above. 
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2.3  Reentry  Optimal  Control  Formulation 


Summarizing  the  previous  reentry  equations,  the  specific  optimal  control  formulation  for  this  RV 
problem  is  stated  as  follows:  Given  an  initial  position  vector  ([x0,  y0,  z0  ] ) ,  velocity  magnitude 

( F0 ) ,  FPA  (y0),  heading  angle  (^0),  AoA  (a0 ) ,  and  AoB  (o  0 ) ,  find  the  control  history 
( ua ,  uG  )  that  maximizes  the  horizontal  do wnrange  distance  (xf )  under  various  constraints. 


xT  =  [X  ,y  ,z  y  ,Y  £  ,a  ,a  ]  u  =  [ua ,  u0  ]  U  =  {ua  ,w„ 


:  -40 


deg 


<  un 


<  40 


deg. 


Min 

s.t. 


J[x(t),u(t),t0  ,tf]  =  -  xf 

x  =  V  cosy  cos^ 
y  =  V  cosy  sin ^ 
z  =  V  sin  y 


V  = 


y  = 


£  = 


D{V,  z}  •  „ 
m 

LjV1z}__g_ cosy_ 
mV  V 

L{V ,  z}sina 
mV  cosy 


a  =  ua, 
c  =  ua 

(t0,x0,y0 ,  z0  ,V0  ,y  0  4  oOt  0>CT  o)  =  (0,0,0, 125000ft, 5714ft/s,-1.3deg,0,15deg,0) 

(. zf,Vf)=  (500/1, 335ft/s) 

-  25  <  zf  <  8.33  (ft/s) 

(0,-oo  ,0,0,-90deg,-90deg,-10deg,-90deg)<  x_<  (oo  ,oo  ,oo?oo?90deg,90deg,50deg,90deg) 

(-2.5g,0,0)<  ( nzq,Q)<  (2.5g,7001b/ft2,60BTU/ft2 -s) 


The  overall  goal  is  to  maximize  the  downrange  distance,  so  this  is  implemented  by  minimizing 
the  negative  of  the  objective  variable,  xf .  Also,  this  particular  formulation  has  an  inequality 

control  constraint  limiting  the  AoA  and  AoB  rates.  Limits  on  the  states  are  represented  by  the 
inequality  “box  constraints.”  The  infinity  signs  were  not  actually  modeled  as  such,  but  made 
very  large  numbers  in  order  to  shrink  tie  searchable  region.  The  end-point  conditions  were 
defined  for  final  altitude  zf ,  velocity  vf ,  and  sink  rate  tf .  The  path  constraints  of  normal  load 

nz ,  dynamic  pressure  q,  and  heating  rate  q  ,  although  not  modeled  in  this  “proof  of- concept”,  are 
represented  as  inequality  “box  constraints.” 
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2.4  Solving  the  Optimal  Control  Problem 

To  demonstrate  optimality,  first  a  theoretical  analysis  is  performed  based  on  optimal  control 
theory.  To  demonstrate  the  necessary  conditions  needed  for  optimality  the  first  step  requires  the 
formulation  of  the  Hamiltonian: 

H(k,x,u,  t)  =  F(x,u,t)+XT f(x,u,  t)  (12) 

where  f(  )  is  the  Lagrange  cost  and  /(•)  is  the  vector  field  for  the  right  hand  side  of  the 
differential  equations  of  motion.  Therefore, 


XT  = 


xf  = 


X 

K 

K 

Xy 

\ 

h 

K 

x 


H()  =  Xx  [V  cosy  cos^]+  X  [V  cosy  sinl;  ]  +  X.  [VsinyJ  +  X^ 


Vt 


(13) 


(14) 


P0e  z“  Srey  2Cd  {a }  -  g  sin  y 
2m 


+  Xy 


^-P  ,e^Srey2CL{a}-gC-^- 
2m  V 


^■Poe  7"^reX  2C/.{a}sina 


mV  cosy 


+  Xxua  +Kuo 


x  =-^-  =>  RHS  of  dynamicequations  ( \  5) 

The  Hamiltonian  Minimization  Condition  (HMC)  is  based  on  Pontryagin’s  Minimum  Principle 
such  that  the  optimal  control  must  minimize  the  Hamiltonian  with  respect  to  control.  For  this 
problem  the  control  is  subject  to  an  inequality  constraint  so  the  Karush- Kuhn- Tucker  (KKT) 
Theorem  is  applied  by  taking  the  Lagrangian  of  the  Hamiltonian: 


(16) 


where  \i  is  a  KKT  multiplier  and  h  is  the  control  constraint  vector.  The  appropriate  necessary 
condition  is: 


dH  _  dH  (  dh  Y 

du  du  ydMj 


p  =  0 


(17) 
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and  substituting  (14)  and  the  controls  into  (17),  the  HMC  becomes 


4+ft  -<>, 
4  +  ft  =0 


4  =  -ft 
4  =  -ft 


(18) 


Also,  the  multiplier- constraint  pair  must  satisfy  the  following  KKT  Complimentarity  Conditions 
(CC): 


if  uao=- 40 

=  0  if  -  40  <  uao  <40 
>0  if 


u-< 


u  =40 

a, a 


-40  if  p  <  0  a  >0  (19) 

40  if  p  >  0  =>  X  <  0 


To  determine  the  final  value  of  the  Hamiltonian,  the  Endpoint  Lagrangian,  given  as, 

E(\),xf,tf):=E(xf,tf)+X)Te(xf,tf)^-xf  +vz(zf  ~  zf)+Vv  (Vf - Vf )  (20) 


is  substituted  into  the  Hamiltonian  Value  Condition  (HVC): 


dE 


H(tf)  + - =  0  =>H(tf)=  0 

at , 


(21) 


This  indicates  that  the  final  value  for  the  Hamiltonian  should  be  approximately  zero  for  this 
problem.  Also,  applying  the  Terminal  Transversality  Conditions  (TTC),  gives  some  indication 
of  the  final  value  of  the  dual  variables  that  can  be  used  later  to  confirm  numerical  results. 

'-f 


.  dE 
h.(t/)=T- 

r)x. 


0 

0 

0 

0 

0 


(22) 


Likewise,  the  Hamiltonian  Evolution  Equation  (HEE)  is  used  to  indicate  the  nature  of  the 
Hamiltonian  with  respect  to  time  such  that: 


dt 


dt 


Here,  the  Hamiltonian  is  constant  with  respect  to  time.  Combining  HEE  (23)  with  HVC  (21),  the 
Hamiltonian  should  be  0  for  all  time. 
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The  above  analysis  is  used  throughout  this  study  to  verify  that  the  numerical  results  satisfy  the 
necessary  KKT  conditions  for  optimality.  For  example,  the  co-state  values  at  the  nodes, 
provided  by  the  Covector  Mapping  Theorem  in  the  Legendre  Pseudospectral  Method,  can  be 
substituted  into  the  necessary  conditions  above  to  evaluate  optimality. 

3.  Sampled-Data  Feedback  Control 

The  architecture  overview  for  the  off-line  and  on-line  trajectory  generation  is  shown  in  Fig.l 
with  the  basic  premise  of  the  sampled- data  closed- loop  RV  guidance  and  control  portrayed  in 
Figures  2  &  3. 


Mission  Objective 
(cost  &  constraints) 


Fig.  1:  Off-Line  and  On-Line  Trajectory  Generation  Control  Architecture 

The  purpose  of  the  architecture  shown  in  Fig.l  is  to  illustrate  the  overall  systems  use  of  an  off¬ 
line  optimal  trajectory  solution,  successive  on-line  optimal  trajectory  solutions,  and  a  simple 
Runge-Kutta  propagation  scheme  to  integrate  the  equations  of  motion  in  order  to  determine  the 
current  vehicle  state  vector.  The  flowchart  for  the  actual  control  algorithm  implemented  in  this 
work  is  shown  in  Fig.2. 
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The  initial  guidance  command  (i.e.  the  reference  trajectory),  is  generated  off-line  by  computing 
the  open- loop  optimal  control  using  DIDO  software  to  provide  the  initial  control  signal,  (ri0,  <|>0) 

and  an  estimate  of  the  final  downrange  distance,  xf .  The  closed-  loop  optimal  control  is  then 
solved  on-line  by  using  the  off-line  solution  as  the  start-up  values  to  a  new  open- loop  optimal 
control  problem  with  sampled  data  fed  back  from  the  previous  DIDO  run.  As  the  next  DIDO  run 
is  in  progress,  the  equations  of  motion  are  propagated  in  real-time  from  the  previously  measured 
state  vector  as  its  initial  condition  using  the  last  interpolated  control  vector,  or  if  not  available, 
the  appropriate  portion  of  the  nominal  control.  Upon  DIDO  completion,  the  vehicle’s  current 
state  is  provided  from  the  propagation  routine  and  a  new  optimal  trajectory  is  computed  using 
the  same  constraints  as  the  off-line  problem.  The  successive  re- optimization  is  illustrated  in 
Fig.3  to  help  clarify  what  is  happening.  Not  shown  are  the  new  complete  optimal  open-loop 
trajectories  beginning  from  each  successive  initial  condition  (IC).  NOTE:  Propagated  (RK4) 
segments  are  exaggerated  for  illustration  purposes  only. 


IC2  (tl=to+tDIDo) 


To  model  external  disturbances  in  the  form  of  wind,  wind  velocity  components 
(Wx,  Wy,  W_ )  were  augmented  to  the  kinematical  and  dynamical  equations  of  motion  for  the 
closed-  loop  implementation  to  give  the  following  equations 


x  =  V  cosy  cosE,  +  WX 
y  =  V  cosy  sinE,  +  W 
z  =  ysiny  +WZ 

V  =  -  D'V,Z'>  -  g  siny  -  ( Wx  cosy  +  Wz  sin y ) 
m 

.  L{V,  z }  g  cosy  1  (■  .  ■  \ 

mV  V  *  z  1 

^  _  L{V,  z}sina 

mV  cosy 


(24) 
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At  lower  altitudes,  these  wind  velocity  components  typically  form  a  wind  shear  that  is  commonly 
represented  by  a  parabolic  gradient  as  a  function  of  altitude.  For  this  “proof  of-concept”  study, 
the  external  disturbances  are  kept  arbitrary;  hence,  details  of  the  wind  gradient  are  not  included 
in  the  model,  but  rather  uniform,  constant  components  are  used  in  the  equations  above  such  that 

the  W  terms  are  zero. 


4.  Results  and  Discussions 

The  maximum  downrange  trajectory  of  a  generic  RV  is  studied  to  illustrate  the  performance  of 
the  above  on-line  closed- loop  control  scheme. 

4.1  Open-Loop  Control  Solution 

For  the  reentry  trajectory,  the  open- loop  maximum- downrange  solution  is  shown  in  Fig.4.  It  is 
seen  that  the  optimal  control  given  by  the  a,<|>  -rate  modulation  drives  the  RV  to  a  maximum 
downrange  of  approximately  1,129,593  ft  (185  nm  or  344  km)  over  a  total  flight  time  of  567 
seconds  and  within  the  allowable  tolerance  of  6.0%  for  the  desired  end-point  conditions. 


Position  (Xe,Ye,Ze) 
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Fig.  4:  Optimal  Open-Loop  States 


a  and  <|)  controls  (rates) 
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Fig.  5:  Optimal  Open-Loop  “Virtual”  Controls 

For  numerical  optimization  methods,  it  is  always  advisable  to  verify  solution  feasibility  and  that 
the  necessary  optimality  conditions  are  indeed  satisfied. 


4.1.1  Demonstration  of  Computational  FEASIBILITY  via  Propagation  Comparison 

The  feasibility  of  the  computational  solution  can  be  validated  by  comparing  the  DIDO  results  to 
the  propagated  states  via  a  separate  ODE  Runge-Kutta  propagator.  By  interpolating  the  values 
of  the  control  function,  u(f),  at  the  LGL  points  and  then  integrating  the  differential  dynamical 
equations,  x=f(x,  4  ),t),  via  MATLAB’s  ode45  solver,  a  comparison  of  error  norms  can  be  made 
with  the  DIDO  trajectory  results.  Results  showed  that  the  open- loop  system  response  does 
satisfy  the  end-point  conditions  within  an  acceptable  error  range.  There  is  some  “norm”  error 
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between  the  DIDO  states  and  the  ode45 -propagated  states;  however,  this  doesn’t  necessarily 
invalidate  the  feasibility  of  the  computational  solution.  There  is  an  obvious  tradeoff  between 
accuracy  and  computational  execution  time.  According  to  Ref.  [24],  solutions  using  greater  than 
60  nodes  (for  a  similar  problem)  will  provide  acceptable  results,  but  at  the  cost  of  significantly 
more  execution  time.  One  way  to  improve  the  accuracy  is  to  increase  the  number  of  nodes  or  to 
relax  the  endpoint  constraint.  Although  this  model  only  fixes  final  altitude,  final  velocity,  and 
final  sink  rate,  a  real  vehicle  must  arrive  to  the  landing  site  in  a  controllable  attitude  and 
reasonable  approach  speed.  As  such,  constraining  the  vehicle’s  final  attitude  could  be  included 
in  the  final  conditions.  Since  the  focus  of  this  paper  is  to  demonstrate  successful  implementation 
of  the  optimal  closed- loop  feedback,  the  fastest  execution  time  possible  is  sought  within 
acceptable  error  tolerances.  For  more  on  DIDO  performance  for  this  type  of  problem,  refer  to 
Ref.  [24], 

4.1.2  Demonstration  of  Computational  OPTIMALITY  via  Bellman’s  Principle 

Comparing  the  numerical  results  to  the  above  theoretical  analysis  of  the  HYC,  HEE,  HMC,  and 
TTC  conditions  validates  the  optimality  of  the  computational  solution.  The  HVC  stated  in  the 
theoretical  analysis  indicates  that  the  Hamiltonian  should  be  0  at  the  final  time  (i.e.  H(tf)=0). 
From  the  HEE,  it  can  be  shown  that  the  Hamiltonian  is  constant  with  respect  to  time.  Combining 
these  two  conditions,  the  Hamiltonian  should  be  0  for  all  time,  clearly  evident  in  Fig. 6. 
Likewise,  the  Lagrange  multipliers  for  the  dynamics  at  the  final  time  match  the  theoretical  TTC 
results  (not  shown). 


Fig.  6:  Hamiltonian  from  Open-Loop  Solution 


Another  test  to  confirm  computational  optimality  is  to  apply  Bellman’s  Principle  of  Optimality. 
This  principle  essentially  states  that  by  using  any  point  on  the  original  optimal  trajectory  as  an 
initial  condition  to  a  new  problem  (with  all  other  problem  formulation  parameters  the  same) 
should  result  in  the  same  optimal  trajectory  with  the  same  or  better  cost.  Although  it  is  not 
shown  here,  this  method  was  used  to  validate  optimality. 
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4.4  Closed-Loop  Control  Solution 


Although  the  previous  open-loop  solution  results  in  a  feasible  trajectory  and  desired  end-point 
conditions  that  satisfy  the  necessary  conditions  for  optimality,  it  does  not  account  for  any 
external  disturbances  as  would  be  the  case  in  real  applications.  Well  known  is  that  without  some 
form  of  feedback,  disturbances,  in  the  form  of  parameter  uncertainty,  sensor  measurement  errors 
(e.g.  noise),  or  unknown/unpredictable  external  forces/moments,  can  result  in  significantly 
degraded  system  performance.  To  illustrate  the  effects  of  external  disturbances  for  this  reentry 
problem,  a  simulated  wind  gust  was  applied  over  a  period  of  20  sec  beginning  at  200  sec  into  the 
flight.  Sensor  measurement  errors  and  parameter  uncertainty  can  be  simulated  by  assuming  the 
role  of  the  errors  from  the  numerical  propagation.  The  effects  of  the  wind  on  the  open-  loop  (OL) 
trajectory  are  seen  in  Fig.  7.  Fig.  8  shows  the  closed- loop  (CL)  results  for  the  variables  of  interest. 


Time  (sec) 

Fig.  7:  Open-Loop  (OL)  System  Response  with  “Nominal”  Wind  Gust 
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f 

Fig.  8 :  System  Response  with  “Nominal”  Wind  Gust 


To  further  illustrate  how  the  closed- loop  system  can  recover  from  the  effects  of  an  external 
disturbance,  Fig.  10  compares  the  results  of  an  exaggerated  600  ft/sec  (—355  knot)  downburst  of 
wind,  often  referred  to  as  a  “microburst,”  applied  over  a  60  second  period.  It  is  evident  from 
figures  11  and  12  that  the  open- loop  system  cannot  recover  from  the  microburst.  The  vehicle  is 
slammed  into  the  ground  for  the  open- loop  trajectory  (5207  ft),  where  as  the  closed- loop 
trajectory  corrects  for  the  microburst  and  is  able  to  achieve  a  final  altitude  of  490  ft,  within  10  ft 
of  the  desired  final  altitude  and  within  the  allowable  end-point  tolerance. 
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Fig.  9 :  System  Controls  for  “Nominal”  Wind  Gust 
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Fig.  10:  System  Response  to  “Exaggerated”  Wind  Gust 
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Fig.  11:  Effects  on  Altitude  with  “Exaggerated”  Wind  Gust 


Fig.  12:  Effects  on  Velocity  with  “Exaggerated”  Wind  Gust 
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Success  of  this  feedback  method  depends  on  relatively  fast  re- computation  time.  Computation  of 
the  first  off-line,  open- loop  optimal  control  trajectory  takes  1 1  seconds  and  the  subsequent  open- 
loop  optimal  control  updates  are  computed  within  0.60  to  11.5  seconds.  The  feedback 
computation  times  are  shown  over  the  entire  trajectory  in  Fig.13.  These  trajectories  were 
generated  on  a  Dell  Optiplex  Desktop  with  a  Pentium  M,  3.40  Ghz  processor,  and  1.0  GB  of 
RAM. 


Fig.  13:  Feedback  Computation  Times  for  System  with  “Nominal”  Disturbance 


For  this  problem,  there  were  no  error- balls  (e)  around  the  initial  conditions  for  each  successive 
update  problem  as  Pooya  used  in  the  time-optimal  satellite  slew  maneuver  problem  [12],  This 
means  that  it  is  more  likely  that  the  re- optimized  closed- loop  trajectories  will  not  follow  the 
original,  off-line  open- loop  response.  According  to  Bellman’s  Principle,  without  the  presence  of 
disturbances,  the  re-optimized  closed- loop  trajectory  should  follow  the  original  optimal  open- 
loop  trajectory.  By  allowing  an  epsilon- error  ball  around  the  initial  conditions,  there  should  be 
less  distinction  between  the  old  and  new  trajectories;  hence,  the  new  trajectory  is  literally  a 
“neighboring”  extremal.  When  disturbances  are  present,  the  optimal  trajectory  changes  with 
each  re- optimization  step  and  are  feedback  to  the  system.  This  results  in  a  different  control 
trajectory  as  seen  in  Fig.  14.  This  unique  control  history  applied  to  the  system  with  external 
disturbances  is  capable  of  countering  their  effects  and  driving  the  vehicle  to  the  desired  end  point 
conditions,  primarily  altitude  and  velocity  in  this  problem. 

A  potential  drawback  for  this  approach  has  to  do  with  the  fact  that  the  optimal  controls,  at  each 
successive  re- optimization  step,  can  essentially  be  reset  to  some  drastically  different  value  within 
the  control  constraint  bounds  as  opposed  to  the  states  that  must  start  at  the  previous  position 
vector.  This  “jump”  in  the  controls  is  illustrated  in  Fig.  14  and  can  be  resolved  by  placing  a 
similar  tolerance  or  e  -error-ball  on  the  initial  control  vector  for  each  re- optimization  problem, 
such  as  |«(0)J+I  —  w* | <e  ,  where  e  is  some  predefined  tolerance. 


19 


7&/d?S^<7SM'/Z%S 


O'a’/K/^^ 


Fig.  14:  Example  of  Successive  Re-Optimization  with  Control  “Jumps” 


Another  potential  problem  is  that  of  convergence.  If  for  some  reason,  one  of  the  re- optimization 
steps  does  not  converge,  it  can  crash  the  entire  closed- loop  process;  however,  additional  logic 
can  be  built  into  the  feedback  algorithm  to  prevent  the  convergence  issue  from  crashing  the 
system.  Such  logic,  as  previously  mentioned,  may  include  using  the  previous  “good”  solution 
for  some  finite  time  and  then  attempting  a  new  re- optimization.  In  the  event  of  frequent  or 
repetitive  non- convergence  issues,  the  only  option  may  be  to  revert  back  to  using  the  open-loop 
response  for  the  remaining  trajectory.  Of  course  this  would  only  be  practical  if  the  remaining 
time-to-  go  is  relatively  short  and  there  are  minimum  uncertainties  and/or  disturbances  during  this 
time.  If  not,  at  least  a  “hard”  crash  is  prevented  and  the  possibility  of  a  feasible  trajectory  still 
exists. 

5.  Conclusions 

In  this  paper,  a  sampled- data  feedback  control  law,  constructed  from  successive  optimal  open- 
loop  solutions,  was  used  to  generate  online,  optimal  trajectories  for  the  reentry  of  a  generic 
reusable  launch  vehicle.  It  was  shown  that  rapid  re- computation  of  the  open- loop  optimal 
control  can  effectively  be  used  for  online  trajectory  generation  in  the  presence  of  various 
disturbances  and  uncertainties.  A  significant  advantage  of  this  control  scheme  is  that  it  does  not 
require  any  advance  knowledge  of  feedback  computation  times;  hence,  no  prediction  method  is 
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required.  As  demonstrated,  solutions  to  difficult  problems  that  may  require  feedback  such  as  on¬ 
line  trajectory  generation  in  the  face  of  disturbances  are  now  possible. 


Acknowledgment 

The  authors  would  like  to  extend  their  appreciation  to  Dr.  Pooya  Sekhavat,  visiting  post-doc 
research  assistant  at  the  Guidance,  Navigation,  and  Control  (GNC)  Laboratory,  Naval 
Postgraduate  School,  for  his  help  with  implementing  and  debugging  the  sampled-data  feedback 
algorithm.  His  efforts  are  greatly  appreciated. 


References 

1.  Harpold,  J  C.,  and  Graves,  C.A.,  “Shuttle  Entry  Guidance,”  The  Journal  of  Astronuatical  Sciences ,  Vol.  27, 
No.  3,  pp. 239-268,  Jul-Sep  1979. 

2.  Oppenheimer,  M.W.,  and  Doman,  D.B.,  “Reconfigurable  Control  Design  for  the  X-40A  with  In-Flight 
Simulation  Results,”  Proceedings  of  the  2005  AIAA  Guidance,  Navigation,  and  Control  Conference ,  AIAA 
Paper  No.  2004-5017,  Aug  2004. 

3.  Oppenheimer,  M.W.,  and  Doman,  D.B.,  “Reconfigurable  Inner  Loop  Control  of  a  Space  Maneuvering 
Vehicle,”  Proceedings  of  the  2003  AIAA  Guidance,  Navigation,  and  Control  Conference ,  AIAA  Paper  No. 
2003-5358,  Aug  2003. 

4.  Bolender,  M.A.,  and  Doman,  D.B.,  Nonlinear  Control  Allocation  Using  Piecewise  Linear  Functions,” 
Journal  of  Guidance,  Control,  and  Dynamics,  Vol. 27,  No.  6,  pp. 1017-1027,  2004. 

5.  Doman,  D.B.,  and  Oppenheimer,  M.W.,  “Integrated  Adaptive  Guidance  and  Control  for  Space  Access 
Vehicles,  Volume  1:  Reconfigurable  Control  Law  for  X-40A  Approach  and  Landing,”  AFRL  IAG&C 
Technical  Report ,  Wright-Patterson  AFB,  OH,  2004. 

6.  Schierman,  J.D.,  Hull,  J.R,  and  Ward,  D.G.,  “On-Line  Trajectory  Command  Reshaping  for  Reusable 
Launch  Vehicles,”  Proceedings  of  the  2003  AIAA  Guidance,  Navigation,  and  Control  Conference ,  AIAA 
Paper  No.  2003-5439,  Aug  2003. 

7.  Schierman,  J.D.,  Hull,  J.R.,  and  Ward,  D.G.,  “Adaptive  Guidance  with  Trajectory  Reshaping  for  Reusable 
Launch  Vehicles,”  Proceedings  of  the  2002  AIAA  Guidance,  Navigation,  and  Control  Conference ,  AIAA 
Paper  No.  2002-4458,  Aug  2002. 

8.  Shaffer,  P.J.,  Ross,  I.M.,  Oppenheimer,  M.W.,  Doman,  D.B.,  “Optimal  Trajectory  Reconfiguration  and 
Retargeting  for  a  Reusable  Launch  Vehicle,”  Proceedings  of  the  2005  AIAA  Guidance,  Navigation,  and 
Control  Conference ,  AIAA  Paper  No.  2005-4168,  Aug  2005. 

9.  Schierman,  J.D.,  Hull,  J.R.,  “In-Flight  Entry  Trajectory  Optimization  for  Reusable  Launch  Vehicles,” 
Proceedings  of  the  2005  AIAA  Guidance,  Navigation,  and  Control  Conference ,  AIAA  Paper  No.  2005- 
6434,  Aug  2005. 

10.  Carson,  J.M.,  Epstein,  M.S.,  MacMynowski,  D.G.,  and  Murray,  R.M.,  “Optimal  Nonlinear  Guidance  With 
Inner-Loop  Feedback  for  Hypersonic  Re-Entry,”  Paper  submitted  to  American  Control  Conference ,  2005. 

11 .  Milam,  M.B.,  Mushambi,  K.,  and  Murray,  R.M.,  “A  New  Computational  Approach  to  Real-Time 
Trajectory  Generation  for  Constrained  Mechanical  Systems,”  Conference  on  Decision  and  Control ,  2000. 

12.  Pooya,  S.,  Fleming,  A.,  and  Ross,  I.M.,  “Time  -Optimal  Nonlinear  Feedback  Control  for  the  NPSAT  1 
Spacecraft,”  Proceedings  of  the  2005 IEEE/ASME ,  Jul  2005. 

13.  Schierman,  J.D.,  Ward,  D.G.,  Monaco,  J.F.,  and  Hull,  J.R.,  “Reconfigurable  Guidance  Approach  for 
Reusable  Launch  Vehicles,”  Proceedings  of  the  2001  AIAA  Guidance,  Navigation,  and  Control 
Conference ,  AIAA  Paper  No.  2001-4429,  Aug  2001 . 

14.  Pesch,  H.J.,  “Off-Line  and  On-Line  Computation  of  Optimal  Trajectories  in  the  Aerospace  Field,”  Prepared 
for  the  12th  Course  in  Applied  Mathematics  in  the  Aerospace  Field  of  the  International  School  of 
Mathematics,  Ettore  Majorana  Centre  for  Scientific  Culture,  Sicily,  Sept  1991. 

15.  Ross,  I.M.,  and  Fahroo,  F.,  “Legendre  Pseudospectral  Approximations  of  Optimal  Control  Problems,” 
Lecture  Notes  in  Control  and  Information  Sciences ,  Vol.  295,  Springer- Verlag,  new  York,  pp. 327-342, 
2003. 


21 


16.  Ross,  I.M.,  and  Fahroo,  F.,  “Issues  in  the  Real-Time  Computation  of  Optimal  Control,”  Mathematical  and 
Computer  Modeling,  An  International  Journal,  Vol.  40,  Pergamon  Publication,  to  appear. 

17.  Ross,  I.M.,  and  Fahroo,  F.,  “Pseudospectral  Knotting  Methods  for  Solving  Optimal  Control  Problems,” 
Journal  of  Guidance,  Control,  and  Dynamics,  Vol.  27,  No.  3,  pp. 397 -405,  2004. 

18.  Ross,  I.M.,  Fahroo,  F.,  and  Gong,  Q.,  “A  Spectral  Algorithm  for  Pseudospectral  Methods  in  Optimal 
Control,”  Proceedings  of  the  6th  IASTED  International  Conference  on  Intelligent  Systems  and  Control, 
Honolulu,  HI,  2004. 

19.  Ross,  I.M.,  and  Fahroo,  F.,  “User’s  Manual  for  DIDO  2002:  A  MATLAB  Application  Package  for 
Dynamic  Optimization,”  NPS  Technical  Report  AA-02-002,  Department  of  Aeronautics  and  Astronautics, 
Naval  Postgraduate  School,  Monterey,  CA,  June  2002. 

20.  Josselyn,  S.  and  Ross,  I.M.,  “Rapid  Verification  Method  for  the  Trajectory  Optimization  of  Reentry 
Vehicles,”  Journal  of  Guidance,  Control,  and  Dynamics,  Vol.26,  No.  3,  pp. 505-508,  2002. 

21 .  Strizzi,  J.,  Ross,  I.M.,  and  Fahroo,  F.,  “Towards  Real-Time  Computation  of  Optimal  Controls  for  Nonlinear 
Systems,”  Proceedings  of  the  2002  AIAA  Guidance,  Navigation,  and  Control  Conference,  AIAA  Paper 
No.  2002-4945,  Aug  2002. 

22.  Regan,  F.J.,  and  Anandakrishnan,  S.M.,  Dynamics  of  Atmospheric  Re-Entry,  AIAA  Education  Series, 
Washington  D.C.,  1993. 

23.  Betts,  J.T.,  Practical  Methods  for  Optimal  Control  Using  Nonlinear  Programming,  Society  for  Industrial 
and  and  Applied  Mechanics  (SIAM),  Philadelphia,  PA  2001. 

24.  Shaffer,  Patrick  J.,  Optimal  Trajectory  Reconfiguration  and  Retargeting  for  the  X-33  Reusable  Launch 
Vehicle,  Thesis,  Naval  Postgraduate  School,  Monterey,  CA,  2004. 


22 


